Designing empirical experiments to compare interactive multiobjective optimization methods

Interactive multiobjective optimization methods operate iteratively so that a decision maker directs the solution process by providing preference information, and only solutions of interest are generated. These limit amount information considered in each iteration support learning about trade-offs. Many interactive have been developed, they differ technical aspects type used. Finding most appropriate method for problem to be solved is challenging, supporting selection crucial. Published research lacks on conducted experiments’ specifics (e.g. questions asked), making it impossible replicate them. We discuss challenges conducting experiments offer realistic means compare methods. propose novel questionnaire experimental design and, as proof concept, apply them comparing two also develop user interfaces these introduce sustainability with multiple objectives. The proposed setup reusable, enabling further experiments.